Sensor fusion display evaluation using information integration models in enhanced/synthetic vision applications

Based on existing integration models in the psychological literature, an evaluation framework is developed to assess sensor fusion displays as might be implemented in an enhanced/synthetic vision system. The proposed evaluation framework for evaluating the operator's ability to use such systems is a normative approach: The pilot's performance with the sensor fusion image is compared to models' predictions based on the pilot's performance when viewing the original component sensor images prior to fusion. This allows for the determination as to when a sensor fusion system leads to: poorer performance than one of the original sensor displays, clearly an undesirable system in which the fused sensor system causes some distortion or interference; better performance than with either single sensor system alone, but at a sub-optimal level compared to model predictions; optimal performance compared to model predictions; or, super-optimal performance, which may occur if the operator were able to use some highly diagnostic 'emergent features' in the sensor fusion display, which were unavailable in the original sensor displays

ATD-2 Integrated Arrival, Departure, and Surface (IADS) Operations

The ATD-2 Integrated Arrival, Departure, and Surface (IADS) traffic management system extends integrated traffic sequencing all the way from the gate to the overhead stream and back again for multi-airport, metroplex environments. NASA and the FAA are developing the IADS system in close coordination with industry partners

Flight Deck Surface Trajectory-Based Operations (STBO): A Four-Dimensional Trajectory (4DT) Simulation

In four-dimensional trajectory (4DT) Surface Trajectory-Based Operations (STBO), aircraft are assigned a conflict-free 4DT which defines an expected location (x,y coordinates) at all times, t, along the taxi route (with altitude, being fixed). These 4DTs afford the highest temporal certainty at all points along the taxi route, and at the departure runway. In the present study, a 4DT flight deck display was presented on the Airport Moving Map (AMM) to support pilot conformance to a 4DT clearance while taxiing under manual control. This pilot-in-the-loop simulation compared the effect of 4DT flight deck display formats on distance from the expected 4DT location, conformance to the displayed tolerance band, eyes-out time, and pilot ratings of safety and workload. In the defined-tolerance display format, a graphical representation of the expected 4DT location, with a distance-based allowable-tolerance band, was depicted on the AMM. Two defined-tolerance band sizes were tested: plus or minus 164 feet and plus or minus 405 feet. In the undefined-tolerance display format, the expected 4DT location was displayed graphically on the AMM, with no indicated allowable-tolerance bounds. Each taxi trial included 4DT speed changes (two or five, per trial) and a range of 4DT taxi speeds. Results showed that the larger (plus or minus 405 feet) defined-tolerance band yielded higher conformance levels than the smaller (plus or minus 164 feet) band, with pilots staying within the specified and displayed conformance bounds more in the larger (99.71%) than the smaller defined-tolerance band (93.37 percent). However, in terms of being able to predict the location of the aircraft compared to the expected 4DT location, the smaller defined-tolerance band resulted in pilots keeping their aircraft closer to the 4DT location, for both average distance and for a given confidence interval (e.g., 95 percent), than either the larger defined-tolerance band or the undefined-tolerance display format. The larger tolerance band yielded more eyes-out-the-window time than the smaller tolerance band. Pilots also rated taxing with the larger tolerance band as safer than the smaller tolerance band

Flight Deck Surface Trajectory-based Operations (STBO): Results of Piloted Simulations and Implications for Concepts of Operation (ConOps)

The results offour piloted medium-fidelity simulations investigating flight deck surface trajectory-based operations (STBO) will be reviewed. In these flight deck STBO simulations, commercial transport pilots were given taxi clearances with time and/or speed components and required to taxi to the departing runway or an intermediate traffic intersection. Under a variety of concept of operations (ConOps) and flight deck information conditions, pilots' ability to taxi in compliance with the required time of arrival (RTA) at the designated airport location was measured. ConOps and flight deck information conditions explored included: Availability of taxi clearance speed and elapsed time information; Intermediate RTAs at intermediate time constraint points (e.g., intersection traffic flow points); STBO taxi clearances via ATC voice speed commands or datal ink; and, Availability of flight deck display algorithms to reduce STBO RTA error. Flight Deck Implications. Pilot RTA conformance for STBO clearances, in the form of ATC taxi clearances with associated speed requirements, was found to be relatively poor, unless the pilot is required to follow a precise speed and acceleration/deceleration profile. However, following such a precise speed profile results in inordinate head-down tracking of current ground speed, leading to potentially unsafe operations. Mitigating these results, and providing good taxi RTA performance without the associated safety issues, is a flight deck avionics or electronic flight bag (EFB) solution. Such a solution enables pilots to meet the taxi route RTA without moment-by-moment tracking of ground speed. An avionics or EFB "error-nulling" algorithm allows the pilot to view the STBO information when the pilot determines it is necessary and when workload alloys, thus enabling the pilot to spread his/her attention appropriately and strategically on aircraft separation airport navigation, and the many other flight deck tasks concurrently required. Surface Traffic Management (STM) System Implications. The data indicate a number of implications regarding specific parameters for ATC/STM algorithm development. Pilots have a tendency to arrive at RTA points early with slow required speeds, on time for moderate speeds, and late with faster required speeds. This implies that ATC/STM algorithms should operate with middle-range speeds, similar to that of non-STBO taxi performance. Route length has a related effect: Long taxi routes increase the earliness with slow speeds and the lateness with faster speeds. This is likely due to the" open-loop" nature of the task in which the speed error compounds over a longer time with longer routes. Results showed that this may be mitigated by imposing a small number oftime constraint points each with their own RTAs effectively tuming a long route into a series of shorter routes - and thus improving RTA performance. STBO ConOps Implications. Most important is the impact that these data have for NextGen STM system ConOps development. The results of these experiments imply that it is not reasonable to expect pilots to taxi under a "Full STBO" ConOps in which pilots are expected to be at a predictable (x,y) airport location for every time (t). An STBO ConOps with a small number of intermediate time constraint points and the departing runway, however, is feasible, but only with flight deck equipage enabling the use of a display similar to the "error-nulling algorithm/display" tested

Extending Validated Human Performance Models to Explore NextGen Concepts

To meet the expected increases in air traffic demands, NASA and FAA are researching and developing Next Generation Air Transportation System (NextGen) concepts. NextGen will require substantial increases in the data available to pilots on the flight deck (e.g., weather,wake, traffic trajectory predictions, etc.) to support more precise and closely coordinated operations (e.g., self-separation, RNAV/RNP, and closely spaced parallel operations, CSPOs). These NextGen procedures and operations, along with the pilot's roles and responsibilities, must be designed with consideration of the pilot's capabilities and limitations. Failure to do so will leave the pilots, and thus the entire aviation system, vulnerable to error. A validated Man-machine Integration and design Analysis System (MIDAS) v5 model was extended to evaluate anticipated changes to flight deck and controller roles and responsibilities in NextGen approach and Land operations. Compared to conditions when the controllers are responsible for separation on decent to land phase of flight, the output from these model predictions suggest that the flight deck response time to detect the lead aircraft blunder will decrease, pilot scans to the navigation display will increase, and workload will increase

A Concept of Operations for Far-Term Surface Trajectory-Based Operations (STBO)

The goal of this far-term STBO (Surface Trajectory-Based Operations) ConOps (Concept of Operations) is to increase the efficiency and predictability of airport surface operations, and reduce the environmental impact, by incorporating a time-based component to surface operations. In the far-term NextGen timeframe, airport surface operations will transition from current-day first-come, first-served operations, to strategically scheduled operations in which pilots are recruited as active participants in meeting the precise time-based goals of NextGen surface operations. The far-term STBO concept includes two-phases. Phase 1 introduces time-based traffic flow constraint points, which divide the taxi route into segments with an assigned Required Time of Arrival (RTA). This Phase 1 approach provides temporal certainty only near the traffic flow constraint points, but not in between. Minimal augmentations to the flight deck are required to support required time of arrival (RTA) management. Phase 2 further increases precision and efficiency by introducing full four-dimensional (4D) trajectories, with an x-y location for all times t. This phase assumes adoption of advanced flight deck equipage enabling higher temporal precision sufficient to support aircraft conformance to 4D trajectories. This allows more precision and less temporal uncertainty at all times along the route

Evaluating Nextgen Closely Spaced Parallel Operations Concepts with Validated Human Performance Models: Flight Deck Guidelines

The objectives of the current research were to develop valid human performance models (HPMs) of approach and land operations; use these models to evaluate the impact of NextGen Closely Spaced Parallel Operations (CSPO) on pilot performance; and draw conclusions regarding flight deck display design and pilot-ATC roles and responsibilities for NextGen CSPO concepts. This document presents guidelines and implications for flight deck display designs and candidate roles and responsibilities. A companion document (Gore, Hooey, Mahlstedt, & Foyle, 2013) provides complete scenario descriptions and results including predictions of pilot workload, visual attention and time to detect off-nominal events

Flight-Deck Surface Trajectory-Based Operations

The results of three piloted simulations investigating flight-deck surface trajectory-based operations (STBO) are presented. Commercial transport pilots were given taxi clearances with time and speed components on the primary flight display and were required to taxi to the departing runway or intermediate intersections. Results show that when pilots were provided with speed-only taxi clearances, pilots either had poor required time of arrival (RTA) conformance with acceptable estimates of attentional distribution and safety, or had good RTA conformance with unacceptable attentional distribution and safety estimates. A flight-deck error-nulling algorithm/display allowed pilots to conform accurately with taxi RTA clearances while maintaining safety. Results are discussed in terms of pilot multitasking in the busy airport surface operations environment

DataComm in Flight Deck Surface Trajectory-Based Operations

The purpose of this pilot-in-the-loop aircraft taxi simulation was to evaluate a NextGen concept for surface trajectory-based operations (STBO) in which air traffic control (ATC) issued taxi clearances with a required time of arrival (RTA) by Data Communications (DataComm). Flight deck avionics, driven by an error-nulling algorithm, displayed the speed needed to meet the RTA. To ensure robustness of the algorithm, the ability of 10 two-pilot crews to meet the RTA was tested in nine experimental trials representing a range of realistic conditions including a taxi route change, an RTA change, a departure clearance change, and a crossing traffic hold scenario. In some trials, these DataComm taxi clearances or clearance modifications were accompanied by preview information, in which the airport map display showed a preview of the proposed route changes, including the necessary speed to meet the RTA. Overall, the results of this study show that with the aid of the RTA speed algorithm, pilots were able to meet their RTAs with very little time error in all of the robustness-testing scenarios. Results indicated that when taxi clearance changes were issued by DataComm only, pilots required longer notification distances than with voice communication. However, when the DataComm was accompanied by graphical preview, the notification distance required by pilots was equivalent to that for voice

A Validated Set of MIDAS V5 Task Network Model Scenarios to Evaluate Nextgen Closely Spaced Parallel Operations Concepts

The Closely Spaced Parallel Operations (CSPO) scenario is a complex, human performance model scenario that tested alternate operator roles and responsibilities to a series of off-nominal operations on approach and landing (see Gore, Hooey, Mahlstedt, Foyle, 2013). The model links together the procedures, equipment, crewstation, and external environment to produce predictions of operator performance in response to Next Generation system designs, like those expected in the National Airspaces NextGen concepts. The task analysis that is contained in the present report comes from the task analysis window in the MIDAS software. These tasks link definitions and states for equipment components, environmental features as well as operational contexts. The current task analysis culminated in 3300 tasks that included over 1000 Subject Matter Expert (SME)-vetted, re-usable procedural sets for three critical phases of flight; the Descent, Approach, and Land procedural sets (see Gore et al., 2011 for a description of the development of the tasks included in the model; Gore, Hooey, Mahlstedt, Foyle, 2013 for a description of the model, and its results; Hooey, Gore, Mahlstedt, Foyle, 2013 for a description of the guidelines that were generated from the models results; Gore, Hooey, Foyle, 2012 for a description of the models implementation and its settings). The rollout, after landing checks, taxi to gate and arrive at gate illustrated in Figure 1 were not used in the approach and divert scenarios exercised. The other networks in Figure 1 set up appropriate context settings for the flight deck.The current report presents the models task decomposition from the tophighest level and decomposes it to finer-grained levels. The first task that is completed by the model is to set all of the initial settings for the scenario runs included in the model (network 75 in Figure 1). This initialization process also resets the CAD graphic files contained with MIDAS, as well as the embedded operator models that comprise MIDAS. Following the initial settings, the model progresses to begin the first tasks required of the two flight deck operators, the Captain (CA) and the First Officer (FO). The task sets will initialize operator specific settings prior to loading all of the alerts, probes, and other events that occur in the scenario. As a note, the CA and FO were terms used in developing this model but the CA can also be thought of as the Pilot Flying (PF), while the FO can be considered the Pilot-Not-Flying (PNF)or Pilot Monitoring (PM). As such, the document refers to the operators as PFCA and PNFFO respectively